Data storage is most economically provided by a storage system including a redundant array of inexpensive disks (RAID). For servicing a large number of network clients, a server is interposed between the network clients and the storage system for authenticating the client requests and for mapping the client requests to storage addresses in the storage system. For servicing a large number of hosts in a computational grid environment, a data switch is interposed between the hosts and the storage system.
When a storage system provides storage for a large number of network clients or a large number of hosts, there has often been a problem that storage access performance has been limited by the data paths between the server or switch and the storage system due to imbalance or overload upon these data paths. This problem has been addressed by various manual and automatic methods of system configuration and load balancing.
Traditionally, a system administrator has planned I/O bandwidth allocation to the client or host applications so that the I/O load is spread over as many paths as are available in the configuration. At the same time, the system administrator has considered the individual storage requirements for each client or host application. Often a significant load imbalance would occur in this environment, causing erratic application performance. Then the system administrator would re-allocate the client or host applications to different paths in order to re-balance the I/O load.
More recently, file servers have been provided with an automatic and dynamic load balancing facility for distributing client I/O requests across all available paths from the server to the storage devices that may service the I/O requests. For example, EMC Corporation has sold the PowerPath® load balancing facility in the form of a software layer between the file system and the SCSI device driver in the file server. The PowerPath® load balancing facility directs each I/O request to a selected port of the storage system based on one of four pre-selected load balancing policies. The system administrator can chose an adaptive policy (the default polity), a round-robin policy, a least I/Os policy, or a least-blocks policy. The adaptive policy is to select the path on the basis of path load (queue with the shortest depth in time) and device priority. The round-robin policy selects the next available path in rotation. The least I/Os policy selects the path with the fewest pending I/O requests. The least blocks policy selects the path with the fewest pending blocks of I/O data.
Although the PowerPath® load balancing facility has been very successful for balancing the load of a single file server upon a data storage system, problems often arise when a data storage system receives I/O requests from more than one file server or host. Without coordination between all of the file servers or hosts feeding I/O requests to the data storage system, load balancing locally at each file server or host based on the local activity at the file server or host may not achieve load balancing globally across all of the ports or storage devices of the data storage system. A loss of host performance is often noticed when file servers or hosts are added to the data processing system. The host performance may be decreased further by an increase in the number of pipes accessing the storage if there is limited network throughput, for example, in a system using 1 gigabit Ethernet pipes.
Loss of performance due to load balancing locally at each file server or host based on activity at the file server or host may become very significant when file servers or hosts share access to the same RAID group, file system, or file. In this case, there may be no increase in activity at any one file server or host if there is an imbalance in loading on certain ports of the storage system due to concurrent access to the shared RAID group, file system, or file. This problem may often arise in a computational grid environment for scientific or engineering applications that use a large number of hosts for parallel processing and use the storage system for passing intermediate results between the hosts. The problem is the same for various storage access protocols such as SCSI, SCSI over IP (iSCSI), InfiniBand, and Fibre Chanel (FC). The problem is aggravated when the hosts use virtual machines such as VMware (Trademark) virtual machines, in which case the unbalance is ported inside the host between multiple virtual machines.